Method for encoding of a video stream

ABSTRACT

A temporal sequence of pictures is generated in a method for encoding of a first video stream. To do so, a synchronization signal is used which is derived from a second video stream independently of the first video stream, or the encoding of a second video stream independent of the first video stream is based on the same principle as for the encoding of the first video stream.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is the United States nation phase under 35 U.S.C. §371of PCT International Patent Application No. PCT/EP2010/004543, filed onJul. 23, 2010.

BACKGROUND OF THE INVENTION

Field of the Invention

Embodiments relate to a method for encoding of a video stream, a methodfor mixing of two video streams, a method to implement a videoconference and a device for implementing or supporting such methods.

Background of the Related Art

Methods for video encoding, that is, for encoding of video data streams,are already being widely used today in many areas of engineering. In thecase of video conference systems, the video streams of severalparticipants are usually combined (“mixed”) into one single videostream. In this kind of combining or mixing, an encoded output videostream is generated from two encoded input video streams, for example,for common display of the two video streams. A method of this kind isdescribed, for example, in WO 2009/049974 A2.

BRIEF SUMMARY OF THE INVENTION

Embodiments specify a method for encoding of a video stream which can beused for such applications, in particular in conjunction with videoconferences.

Accordingly, embodiments provide that in the generation of the encodedsequence of pictures, a synchronization signal is used which is derivedfrom a second video stream independently of the first video stream, orthe encoding of a second video stream independently of the first videostream is based on the same principle as for the encoding of the firstvideo stream.

BRIEF DESCRIPTION OF THE FIGURES

The figures show:

FIG. 1: A schematic view of one preferred exemplary embodiment of aninvented method for encoding of a video stream;

FIG. 2: A schematic view of an additional, preferred exemplaryembodiment of an invented method for encoding of a video stream;

FIG. 3: A first video stream with a first prediction structure with aso-called IPPP-encoding;

FIG. 4: A second video stream with a second prediction structure with aso-called IPPP-encoding;

FIG. 5: an output video stream produced by mixing of the video streamsshown in FIGS. 3 and 4, in which the prediction structures are notsynchronized according to the invention;

FIG. 6: A third video stream with a third “hierarchical” predictionstructure;

FIG. 7: A fourth video stream with a fourth “hierarchical” predictionstructure;

FIG. 8: an output video stream produced by mixing of the video streamsshown in FIGS. 6 and 7, in which the prediction structures are notsynchronized according to the invention;

FIG. 9: The video stream shown in FIG. 6 with a synchronization signal saccording to the invention;

FIG. 10: A video stream modified according to the invention and with ashortened picture group length and with a synchronization signal saccording to the invention;

FIG. 11: an output video stream produced by mixing of the video streamsshown in FIGS. 9 and 10, in which the prediction structures aresynchronized according to the invention;

FIG. 12: A video stream with a “hierarchical” prediction structure inwhich one group with 7 P-frames depends on a non-prediction-encodedI-frame.

FIG. 13: A video stream with a “hierarchical” prediction structure witha picture group length shortened by the synchronization signal saccording to the invention, in comparison to the illustration in FIG.12;

FIG. 14: An output video stream produced by mixing the video streamssynchronized according to the invention and shown in FIGS. 12 and 13.

DETAILED DESCRIPTION OF THE INVENTION

The derivation of a signal or of an information item from a data stream,in particular of a synchronization signal from a video stream, isunderstood to be in particular in connection with the description of thepresent invention, any kind of generation of one such signal or of onesuch information item, in which structural properties of the data streamfrom which the signal or the information item is derived, are used forgeneration of the derived signal or of the derived information item.Since a data stream relates to a time sequence of data or data groups,for example, of pictures, picture points or blocks of picture points,the structure of one such data stream is determined by the structuralproperties of these data or data groups and their association to timepoints. In the exemplary case of a data stream of a temporal sequence ofequivalent data blocks—that is, data blocks constructed according to aparticular pattern—which are each associated with a particular point intime, a synchronization signal could be derived from this data stream,for example by detection of these time points and by generation of asignal which describes these time points. Additional examples for thederivation of a signal or of an information item from a data stream, inparticular of a synchronization signal from a video stream will beprovided in the further course of the description of preferredembodiments of the invention.

Therefore, in the invented method a first video stream is produced bymeans of a synchronization signal which is derived from a second videostream independent of the first video stream, or which indeed is notderived from this second video stream, but is based on the encoding ofthe second video stream in a manner corresponding to the encoding of thefirst video stream. Therefore this synchronization signal can also be anexternal synchronization signal, for example, a simple timing signalwhich is used in a corresponding manner as the basis for encoding of atleast two of the encoded video signals.

In the encoding, that is, in the compression of video streams or ofsequences of moving pictures, the data reduction associated with thecompression is achieved essentially in two ways: Firstly, singlepictures are compressed with a preferably block-based method, forexample, by means of a method known as discrete cosine transformation(DCT). This method roughly corresponds to the known JPEG-standard forsingle pictures. In addition, dependencies (correlations) betweensequential, single pictures in which the picture content often differsvery little, are used for the data reduction. For this purpose,so-called prediction structures are introduced in which preferably threepicture types (also known as frames) are used:

-   -   The frames known as I-frames are saved without using the        correlation between sequential pictures. These frames thus do        not depend on the following or preceding frames. Since only        picture content of this one frame is used for encoding of these        pictures, these pictures are also known as “intra-coded”        pictures. Thus we have the name I-frame.    -   The so-called P-frames are additionally predicted from a        preceding P or I-frame and are thus also known as predicted        frames.    -   And the B-frames get their name because these frames are        bidirectionally interpolated and/or predicted. In contrast to        the P-frames, references to a following P or I-frame may        additionally be present. However, for decoding of a B-frame, the        following referenced frame must already have been decoded, such        that a larger number of saved frames is needed and the total        delay time of the decoding process is often increased.

A time sequence of these picture types characterizes a so-calledprediction structure. Thus in this regard we are dealing with astructural property of a video stream from which preferably asynchronization signal or a corresponding information item can bederived. For example, a synchronization signal of this kind can bederived from the prediction structure of a video stream, wherein thetime points which are allocated to I-frames in this video stream, forexample, are introduced in the synchronization signal. Otherpossibilities for derivation of a synchronization signal from a videosignal will be provided below in the description of preferredembodiments of the invention.

In connection with the description of the present invention, the conceptof encoding (also: encoding, coding) means the digital representation ofa video stream, that is, of a data stream which represents a videosignal, that is, a temporal sequence of digital or digitized pictures,associated preferably with a reduction in the data volume (datacompression, compression). Decoding of this kind of encoded video streamusually produces a data stream which allows a replay or processing ofthe video signals.

In one preferred embodiment of the present invention, the sequence offrames includes prediction-encoded pictures, in particular P-frames, andnon-prediction encoded pictures, in particular I-frames, and thesynchronization signal, is used for synchronization of the positions ofnon-prediction encoded pictures, in particular of I-frames, in the twosequences of pictures of the two independent video streams. In the firstcase that the synchronization signal is derived from the second videostream, the synchronization signal is used preferably to control thepositions of non-prediction-encoded pictures in the first video stream.In the other case, where the synchronization signal is used for encodingof the two video streams in a corresponding manner, the positions ofnon-prediction-encoded pictures in both sequences of pictures iscontrolled in a corresponding manner.

The prediction of pictures makes use of the fact that certain pictureportions in the temporally sequential pictures change only very little,or merely assume a different position in the following picture. Underthese prerequisites, a prediction of future picture content is possiblewith the aid of motion vectors which indicate the shift of pictureportions between sequential pictures. However, usually residualdifferences between the picture block being encoded remain, which thencan be encoded, for example, by means of a discrete cosine transform andsubsequent quantizing.

According to an additional, preferred embodiment, the invention providesthat the synchronization signal is generated by a device for mixing ofthe first and second video stream. Examples of such devices are videoconference systems, in particular the server used in the process, whichmakes available a plurality of encoded video streams to various videoconference participants through subscriber terminals. Thesynchronization signal in this case contains an information bit relatingto the time offset between the positions of non-prediction codedpictures, in particular of I-frames, in the two sequences of pictures ofthe two independent video streams, or is derived from one suchinformation item.

In another preferred embodiment, the synchronization signal contains aninformation item relating to the number of prediction-encoded pictures,in particular the P-frames or the B-frames, which follows anon-prediction encoded picture, in particular after an I-frame, in atleast one of the two video streams until the arrival of an additionalprediction-encoded picture, or is derived from one such informationitem.

The method according to the invention and the various exemplaryembodiments are suitable for mixing two video streams, wherein at leastone of these video streams is or was encoded according to a method ofthe kind described above. Therefore, these methods are also suitable forimplementation of a video conference in which at least two video streamsare mixed according to one of the named methods.

As a preferred embodiment of a method of this kind for implementing avideo conference, the invention provides that upon arrival of anadditional participant in the video conference, said participant's videostream is initially encoded non-synchronized, and that saidparticipant's video stream is synchronized as soon as a device formixing of video streams can generate a synchronization signal accordingto one of the preceding or following claims. Particularly preferred inthis regard is an embodiment of the method in which a device for mixingvideo streams signals a desired prediction structure before or duringthe synchronization.

The present invention can also be implemented by means of an apparatusto implement or to support one of the named methods, said apparatusbeing designed to generate and transmit or to receive or to process asynchronization signal according to one of the described methods.

The invention will be explained in greater detail below based on thefigures and with the aid of preferred, exemplary embodiments.

FIGS. 3, 4 and 5 show the mixing of two video streams with anIPPP-encoding, in which the prediction structures are not synchronizedaccording to the invention. In the video stream illustrated in FIG. 3,the pictures 31, 32, 33, 34, 35, 36, 37 and 38 follow each other intime. Pictures 31 and 35 are not prediction-encoded (“intra-encoded”)I-frames. Pictures 32, 33, 34, 36, 37 and 38 are prediction-encodedP-frames. I-frames 31 and 35 encoded without reference to anotherpicture can be decoded without reference to any other picture. TheP-frames are encoded by means of a prediction of their picture contenton the basis of a preceding picture, and thus can only be decoded withthe aid of the picture content of this preceding picture.

Accordingly for the video stream illustrated in FIG. 4 and composed ofthe I-frames 42 and 46 and the P-frames 41, 43, 44, 45, 47 and 48, withthe difference that the I-frames 42 and 46 appear in the video streamillustrated in FIG. 4, at time points when the P-frames 32 and 36 appearin the video stream illustrated in FIG. 3. For decoding of the P-frame41, knowledge of the picture content of a picture preceding picture 41is required and not illustrated in FIG. 4. Picture 48 is needed fordecoding of a picture (not illustrated in FIG. 4) which follows picture48 in time.

In FIGS. 3 and 4, the individual picture groups (“groups of pictures,”GOPs) indeed do have the same length; however, the starting points ofthe picture groups, namely the I-frames 31, 35, 42 and 46, are shiftedin time with respect to each other. The point in time when thenon-prediction-encoded I-frames 42 and 46 appear in the video streamshown in FIG. 4 corresponds to that point in time in the video datasequence illustrated in FIG. 3, when the prediction encoded P-frames 32and/or 36 appear in the latter video data sequence. Thus when mixing thetwo video streams shown in FIGS. 3 and 4 without a synchronizationaccording to the invention, all pictures in the output video streamshown in FIG. 5 are prediction-encoded P-frames, namely pictures 51 to58. All pictures are connected through references to neighboringpictures.

This phenomenon then means that there are no random access points forthe output video stream, which is a disadvantage for the reliability ofthe method and for its fault tolerance.

In the case of hierarchical encoding, an additional problem appears. Ahierarchical encoding enables a temporal scalability, which makespossible, in part, the realization of improved error-resilient methods.For example, in video streams with temporal scalability, the timebase-plane, that is, the lowest time resolution step, can be easilyshielded in order to prevent uncontrolled error propagation. In contrastthereto, in an IPPP-encoding with the loss of one P-frame, all followingP-frames no longer can be decoded without error.

In the video stream shown in FIG. 6, the P-frames 63 and 67 do notdepend on their respective preceding p-frames 62 and/or 66, but ratheron their respective preceding I-frames 61 and/or 65. But in contrast,the p-frames 64 and 68 depend on their preceding P-frames 63 and/or 67.This applies similarly to the video stream shown in FIG. 7. The P-frames74 and 78 do not depend on their respective preceding P-frames 73 and/or77, but rather on their respective preceding I-frames 72 and/or 76. Butin contrast, the p-frames 71 and 75 depend on their preceding P-frames70 and/or 74, wherein the P-frame 70 is not shown in FIG. 7.

As is shown in FIGS. 6, 7 and in particular in FIG. 8, this hierarchicalprediction structure—when mixing the video streams shown in FIGS. 6 and7—leads to the problem that numerous pictures, namely pictures 83, 84,87 and 88 in the output picture sequence shown in FIG. 8, will havedependencies on several previous pictures, that is, on several referencepictures, which are also known as “multiple references,” which usuallyresults in an increased memory requirement.

For example, picture 83 depends on pictures 81 and 82; picture 84depends on pictures 82 and 83; picture 87 depends on pictures 85 and 86,and picture 88 depends on pictures 86 and 87. These kinds of multiplereferences increase the probability of errors in the decoding and oftenalso increase the expense for encoding and decoding. Moreover, thesemultiple references cannot be reflected in many video coding standards,and the temporal scalability is also lost, which is indicated in FIG. 8by the “?”-symbol. This results in a greater error susceptibility in thedecoding of the output video stream.

But now the invention solves this problem as illustrated in FIGS. 1 and2, through the control of encoding E1 and/or E2 of at least one videodata stream 1 and/or 2 as a function of a synchronization signal s1and/or s12, which is provided in the embodiments shown in FIGS. 1 and 2through a device E12 for mixing of the video data streams 1 and/or 2,and/or of their encoded versions 1′ and/or 2′. The encoders E1 and/or E2encode the video streams 1 and/or 2 and generate the encoded videostreams 1′ and/or 2′, respectively. Then in the embodiment shown in FIG.1, these video streams are sent the device E12 for mixing the two videostreams, whereupon or wherein this device creates the synchronizationsignal s1, which is sent to the device E2 and is used therein forencoding of the video stream 2.

In the exemplary embodiment illustrated in FIG. 1, the synchronizationsignal s1 is sent only to the encoder E2, but not to encoder E1.Nonetheless, synchronization is possible because in this embodiment ofthe invention, the synchronization signal s1 is derived from the videostream 1 and/or 1′. Now the synchronization signal derived from videostream 1 and/or 1′ thus contains information for the synchronization ofthe encoding E2 of the video stream 2, which is derived from thestructural properties of video stream 1′, for example, from itsprediction structure. The device for mixing E12 generates the mixedvideo stream 12 by using the video streams 1′ and 2′ synchronizedaccording to the invention.

In the exemplary embodiment of the invention illustrated in FIG. 2, thesynchronization signal s12 is sent to the two encoders E1 and E2.Therefore, this synchronization signal s12 need not be derived from oneof the two video streams. Instead, this may be related also to anexternal signal, for example, a timing signals which is used by bothencoders E1 and E2—in a corresponding manner—for synchronization. Theexpression “in a corresponding manner” in this regard means that thesynchronization signal is used by both encoders E1 and E2algorithmically in the same manner for encoding of the respective videostreams 1 and 2.

In the method according to the invention, a synchronization signal isused which is derived from a second video stream independently of thefirst video stream, or the encoding of a second video stream independentof the first video stream is based on the same principle as for theencoding of the first video stream. Therefore one essential idea of theinvention is to synchronize the input video streams, preferably theirprediction structure, in order in this manner to generate an improvedoutput video stream during mixing.

For this purpose the invention provides for the control of at least oneof the two encoders such that this kind of synchronizing can occur. Inorder to achieve a synchronization of video streams with a defaultprediction structure, two fundamental means are suitable, and they canbe combined with each other: The signaling of shifts by a centralserver, for example, by a device for mixing the video streams, or theuse of a common temporal basis. Both methods or a combination thereofcan be supplemented by a precision control of the picture repetitionrate.

The device E12, for example, a server which implements the mixing of theinput video streams 1′ and 2′, can calculate the time offset of theinput data streams, for example. In order to eliminate the calculatedoffset by means of the synchronization, this device E12, e.g. a serverin a video conference system, sends an instruction to the video encoderof the corresponding video data source(s) (“end points”) with a promptto reduce the number of pictures in a group of pictures (“GOP”) by thecurrently calculated offset. In an additional embodiment of theinvention, the length of the group of pictures can also be extended, ora combination and/or a mixed form of a shortened or a lengthened groupof pictures can be used. In the event that the length of the group ofpictures of the input video streams is still not equal, then it willalso be transmitted as a new parameter.

This procedure is illustrated, for example, in FIGS. 9, 10 and 11. Thesynchronization signal s of the device E12 for mixing the input videostreams to the encoder for the video stream illustrated in FIG. 10 couldconsist roughly of an instruction to shorten the length of the group ofpictures by one picture. The encoder could then execute this instructionat the next opportunity that results.

For example, FIG. 9 shows a video stream with pictures 91 to 98consisting of two groups of pictures from pictures 91 to 94 and pictures95 to 98. The respective first pictures 91 and 95 of a group of picturesin this example are I-frames, all other pictures 92, 93, 94, 96, 97 and98 are P-frames or p-frames. The difference between the upper-caseletter and the lower-case letter is thus used to indicate that thepictures belong to different levels of the time resolution.

This situation corresponds to the situation shown in FIG. 6. Now inorder to avoid the problems illustrated in FIG. 8, the encoder of thevideo stream illustrated in FIG. 7 will be triggered by thesynchronization signal to shorten the length of the group of pictures.From the video stream shown in FIG. 7, we then have the video streamillustrated in FIG. 10, in which prediction-encoded picture 75 does notfollow prediction-encoded picture 74, but rather in which thenon-prediction encoded I-frame 105 follows the prediction-encodedpicture 104. Therefore, the encoder is caused by the synchronizationsignal to encode picture 105 without reference to a preceding picture,i.e. to generate a non-prediction-encoded I-frame 105.

When mixing these two video streams synchronized according to theinvention and illustrated in FIGS. 9 and 10, the output video streamillustrated in FIG. 11 is generated, in which the multiple references ofpictures 87 and 88 do not appear, as shown in FIG. 8. None of thepictures 116, 117 or 118 is dependent on more than one precedingpicture.

But a group of pictures need not necessarily begin with an intra-frame(I-frame), rather they can also begin with a prediction-encoded picture,as is shown in FIGS. 12, 13 and 14. In this manner we can avoid a shortterm rapid increase in the data rate in the network due to thesimultaneous transmission of I-frames from all senders. In this regard,preferably an information item indicating whether the group of picturesis to begin in an intra-frame, can also be additionally signaled andtransmitted, or it can be integrated into the synchronization signal.

In several preferred exemplary embodiments of the invention, theprediction structure and the intra-frame interval can be signaled to anencoder in the synchronization signal or in addition to thesynchronization signal, as can be seen, for example, in the videostreams illustrated in FIGS. 12 and 13. This is an advantage inparticular when the prediction structure generated by the encoder doesnot coincide with the prediction structure expected by the mixer E12. Insuch cases, for example, the signaling might appear as follows: “I0 p2P1 p2” with “intra-period=8.” The letter symbols therein relate to thepicture type, wherein I stands for the intra-frame picture type, P(“capital P”) stands for the “P-reference picture” type, p (“small p”)stands for the “P-non-reference picture” type. The “intra-period”parameter denotes the temporal scaling step.

In another preferred embodiment of the invention, the information itemcan also have another content, which however, produces a similar orequal behavior of the addressed encoder. One possibility for thespecification would be to instruct the encoder to start the group ofpictures with a particular picture number, or if the lengths of thegroup of pictures do not coincide, to start with a dedicated length ofthe group of pictures. The corresponding instruction might look asfollows: “new group of pictures with group of pictures length equal to xat picture number y.” Calculation of the picture number occurs throughthe server from the shift in the video streams and the delay in thesignaling.

In the latter case it must be assured that the signaling packet reachesthe encoder before the encoding of the picture number for the new groupof pictures. In both the stated cases the signaling can take place, forexample, by means of a protocol for real time control of media streams,preferably by means of the RTP Control Protocol (RTCP).

If a new participant arrives in a video conference, then he can startfirst as non-synchronized with the encoding and sending of the videodata. Thus initially a previously, potentially available synchronicity(equal prediction structure) of the other subscribers will be lost. Butthen the new subscriber will be synchronized preferably as much aspossible as soon as the server can calculate the offset, for example.The desired prediction structure can be signaled in advance to the newparticipant. This can proceed preferably during the establishment of theconnection or through the already described, RTCP-like signaling.

The described exemplary embodiments and those to be presented below canalso be implemented in combination. The signaling herein can comprisethe following elements in general, which can be combined in a suitablemanner:

-   -   A picture offset or a decrease or increase in the group of        pictures length    -   A decision on whether a new group of pictures begins with an        infra-frame    -   The length of the group of pictures    -   The prediction structure which implicitly contains an        information item about the length of the group of pictures    -   The intra-period, that is, the spacing of the intra-pictures

These control elements or parameters are derived preferably from asecond video stream, that is, they are calculated or determined from itsprediction structure or from other structural properties of this videostream. Various examples of this were described above.

The synchronization of the prediction structures can also be attainedthrough a common time basis. Therefore, the invention provides forembodiments in which each end point is first synchronized with areference time basis. This can occur, for example, by means of theso-called Network Time Protocol (NTP). The communications server E12which effects the mixing of the video streams 1′ and 2′ can also beused, for example, as a source for the reference time basis. This kindof situation is illustrated in FIG. 2, for instance.

In one preferred embodiment of the invention, the signaling can proceedsuch that the server sends a prompt at each end point E1 or E2 to startsending of a particular prediction structure at a particular time. Thestarting point is thus calculated preferably from the transmission timeof the data from the end point to the server. This transmission time ofthe data from the end point to the server can be estimated, preferablyas one-half of the Round Trip Time (RTT), for example. Then timing forthe beginning of the new group of pictures can be calculated preferablyas follows:T(new group ofpictures;i)=T(mixing;i)−T(transmission;i)≈T(mixing,i)−RTT/2,

-   -   where i=1, . . . , n, wherein n is the number of end points,        i.e. of the independent video streams being mixed, that is, the        number of conference participants, for example.

Due to the specification of the starting point for a group of picturesand the specification of the prediction structure, the sender cancalculate a fixed mapping between prediction structure and time basis,and then supply a video stream with a synchronized prediction structure.From experimentally confirmed estimations it turns out that the accuracyof the Network Time Protocol (NTP) is about 10 milliseconds here.

Therefore, inaccuracy in of synchronization based on this amounts to amaximum of 20 milliseconds since the end points can deviate in differentdirections, (i.e. they can “advance or delay”). At a picture refreshrate of 25 Hz, this corresponds to an offset of one frame.

As already mentioned, this offset, if present, can be compensated asdescribed above, by the signaling of the shift. Depending on theapplication and the embodiment of the invention, precision control ofthe picture refresh rate can be favorable or desirable. Since the timereferences can diverge at the individual end points, especially withoutthe use of a common time basis, an offset can build up over time, evenfor synchronized video streams and for a formally equal picture refreshrate. In order to counteract an offset of this kind, the picture rate ofone or several end points can preferably be corrected accordingly. Thusthe server preferably sends an instruction to the end point(s) E1 and/orE2 with roughly the following content: “Increase the picture refreshrate by x,” wherein a negative value for x should correspond to areduction in the picture refresh rate.

The correction value x thus can preferably be calculated from thedeviation of the input data stream at the reference time as follows:X=[(target picture rate/estimated picture rate)−1]*100%

At an estimated picture rate which corresponds to the number of receivedpictures in the time interval (thus per time interval).

The described invention, depending on the particular embodiment, willallow the mixing of video streams with a relatively low expense, inparticular in comparison to complete transcoding of the video streamsbeing mixed. And the temporal scalability will be retained.

Thus it is possible to adapt the output video stream with respect to thepicture refresh frequency and the data rate to the requirements of theapplication, and specifically for a simultaneously reducedsusceptibility to error, preferably by means of a separate faultprotection, for example, by means of re-transmissions, for the timebasis layer, i.e. the lowest time resolution step. Complex predictionstructures in the output video stream, which cannot be mapped undercertain circumstances by a video coding standard, can be avoided bymeans of the invention.

The output video stream generated according to the invention often canbe decoded with a smaller memory requirement. An additional delay whichis often unavoidable in conventional methods, can be minimized oreliminated entirely with the invented method since the individual inputvideo streams being mixed are not delayed.

The exemplary embodiments of the invention described above can also becombined with each other in an advantageous manner. However, theinvention is not restricted to the exemplary embodiments explicitlydescribed above. Based on the above description of the invention, anordinary person skilled in the art will be inherently placed into aposition to find and to implement additional, favorable embodiments.

The invention claimed is:
 1. A method for mixing at least two videostreams comprising: generating a first video stream by a first endpoint, the first video stream comprising a first temporal sequence offrames; generating a second video stream by a second end point, thesecond video stream comprising a second temporal sequence of frames; thefirst end point sending the first temporal sequence of frames to amixing device; the second end point sending the second temporal sequenceof frames to the mixing device; the mixing device deriving asynchronization signal from at least one of: the first temporal sequenceof frames, the second temporal sequence of frames, and a timing signal;the mixing device sending the synchronization signal to at least one ofthe first end point and the second end point so that subsequent framesof the first temporal sequence and subsequent frames of the secondtemporal sequence sent to the mixing device are synchronized with eachother via synchronized encoding of the first and second subsequentframes by encoders of the first and second end points; at least one ofthe first end point and second end point receiving the synchronizationsignal; at least one of the first end point and the second end point, inresponse to the received synchronization signal, adjusting encoding ofthe frames of the first video stream and the frames of the second videostream such that subsequent frames of the first temporal sequence sentto the mixing device and subsequent frames of the second temporalsequence sent to the mixing device are synchronized with each other tohave a same predictive structure based on the received synchronizationsignal; the mixing device receiving the subsequent frames of the firsttemporal sequence and the subsequent frames of the second temporalsequence after having sent the synchronization signal and mixes thereceived subsequent frames of the first temporal sequence with thereceived subsequent frames of the second temporal sequence to generate amixed video stream.
 2. The method of claim 1, wherein the synchronizedpredictive structure of the subsequent frames of the first temporalsequence sent to the mixing device and the subsequent frames of thesecond temporal sequence sent to the mixing device is configured so thateach P-frame of a sequence of P-frames in the subsequent frames of thefirst temporal sequence has a position corresponding to a respectiveP-frame of a sequence of P-frames in the subsequent frames of the secondtemporal sequence.
 3. The method of claim 1, wherein the mixing deviceis a server of a video teleconference system, the first end point is asubscriber terminal and the second end point is a subscriber terminal.4. The method of claim 1, wherein the synchronization signal contains aninformation bit identifying a time offset between the positions ofI-frames within the subsequent frames of the first temporal sequencesent to the mixing device or within the subsequent frames of the secondtemporal sequence sent to the mixing device.
 5. The method of claim 1,wherein the synchronization signal contains an information itemidentifying a number of P-frames or number of B-frames to follow anI-frame in at least one of the subsequent frames of the first temporalsequence sent to the mixing device and the subsequent frames of thesecond temporal sequence sent to the mixing device.
 6. The method ofclaim 1 wherein the synchronization signal is only sent to the secondend point and is derived from the first temporal sequence of frames. 7.The method of claim 1 wherein the synchronization signal is sent to boththe first end point and the second end point.
 8. The method of claim 1,wherein the first end point, second end point, and mixing device areelements of a video conference system and wherein the first and secondend points are within a video conference, the method comprising: uponarrival of an additional participant in the video conference, encoding athird video stream from a third end point of the additional participantas non-synchronized for a first time period and subsequentlysynchronizing said third video stream via communications with the mixingdevice after the mixing device generates a synchronization signal forthe third end point for synchronizing encoding of the third video steamwith the encoding of the first video stream and the encoding of thesecond video stream so that subsequent encoding of frames of the thirdvideo stream are encoded to have the same predictive structure as theencoding of the first and second video streams performed by the encodersof the first and second end points.
 9. The method of claim 8, comprisingsignaling before or during synchronization, a desired predictionstructure to an encoder of the third end point.
 10. The method of claim1 wherein the synchronization signal is sent to both the first end pointand the second end point and the synchronization signal is based on thetiming signal.
 11. The method of claim 1 wherein the predictivestructure defines a group of pictures for encoding of at least one ofthe first video stream and the second video stream such that thesubsequent frames of the first temporal sequence sent to the mixingdevice and the subsequent frames of the second temporal sequence sent tothe mixing device have a same length for individual picture groups. 12.The method of claim 1, wherein the mixing device is a central server andthe first end point is a terminal and the second end point is a terminaland wherein the mixed video stream is a lone output of the centralserver for a teleconference involving the first and second end points.13. The method of claim 1, wherein the synchronization signal isconfigured so that the encoder of the first endpoint encodes thesubsequent frames of the first temporal sequence and the encoder of thesecond end point encodes the subsequent frames of the second temporalsequence in a corresponding manner so that I-frames within thesubsequent frames of the first temporal sequence correspond withI-frames within the subsequent frames of the second temporal sequenceand P-frames within the subsequent frames of the first temporal sequencecorrespond with P-frames within the subsequent frames of the secondtemporal sequence.
 14. The method of claim 1, wherein thesynchronization signal is configured so that each of the subsequentframes of the first temporal sequence is dependent on no more than onepreceding frame within the first temporal sequence.
 15. The method ofclaim 14, wherein the synchronization signal is also configured so thateach of the subsequent frames of the second temporal sequence isdependent on no more than one preceding frame within the first temporalsequence.
 16. The method of claim 1, wherein the synchronization signalis also configured to identify that each group of pictures within thesubsequent frames of the first temporal sequence is to start with anI-frame.
 17. A video conference system comprising: a first end point; asecond end point; and a mixing device; the first end point generating afirst video stream comprising a first temporal sequence of frames; thesecond end point generating a second video stream comprising a secondtemporal sequence of frames; the first end point sending the firsttemporal sequence of frames to the mixing device; the second end pointsending the second temporal sequence of frames to the mixing device; themixing device deriving a synchronization signal from at least one of:the first temporal sequence of frames, the second temporal sequence offrames, and a timing signal; the mixing device sending thesynchronization signal to at least one of the first end point and thesecond end point so that subsequent frames of the first temporalsequence and subsequent frames of the second temporal sequence sent tothe mixing device are synchronized with each other via synchronizedencoding of the first and second subsequent frames by encoders of thefirst and second end points; at least one of the first end point andsecond end point receiving the synchronization signal; at least one ofthe first end point and the second end point, in response to thereceived synchronization signal, adjusting encoding such that subsequentframes of the first temporal sequence sent to the mixing device andsubsequent frames of the second temporal sequence sent to the mixingdevice are synchronized with each other to have a same predictivestructure based on the received synchronization signal; the mixingdevice receiving the subsequent frames of the first temporal sequenceand the subsequent frames of the second temporal sequence after havingsent the synchronization signal and the mixing device mixing thereceived subsequent frames of the first temporal sequence with thereceived subsequent frames of the second temporal sequence to generate amixed video stream.
 18. The system of claim 17, wherein the mixingdevice is a server or a central server, the first end point is aterminal and the second end point is a terminal.
 19. The system of claim18, wherein the predictive structure defines a group of pictures forencoding of at least one of the first video stream and the second videostream such that the subsequent frames of the first temporal sequencesent to the mixing device and the subsequent frames of the secondtemporal sequence sent to the mixing device have a same length forindividual picture groups; and wherein the synchronization signal isconfigured so that the encoder of the first endpoint encodes thesubsequent frames of the first temporal sequence and the encoder of thesecond end point encodes the subsequent frames of the second temporalsequence in a corresponding manner so that I-frames within thesubsequent frames of the first temporal sequence correspond withI-frames within the subsequent frames of the second temporal sequenceand P-frames within the subsequent frames of the first temporal sequencecorrespond with P-frames within the subsequent frames of the secondtemporal sequence.
 20. The system of claim 18, wherein thesynchronization signal contains an information bit identifying a timeoffset between the positions of I-frames within the subsequent frames ofthe first temporal sequence sent to the mixing device or within thesubsequent frames of the second temporal sequence sent to the mixingdevice.